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This brief summarizes the findings of Mathematica s 
Valuing Competencies study, which analyzed the 
effeets of students ’high school competencies on their 
postsecondary earnings and educational attainment. 
We estimated the effects of several competency 
measures, including academic achievement (as 
eaptured by test scores) and composite measures of 
leadership skills, sports-related skills, work habits, 
prosocial behavior, and locus of control (a measure of 
students ’ belief that they control their future) . Using 
the National Education Longitudinal Study (NELS), 
a survey of students, parents, and school staff, as well 
as tests administered during high sehool, we estimated 
the fraction of students who benefited most from 
gains in each competency measure and how these 
benefits depended on a student ’s existing strengths 
and weaknesses. 



Debate Over Academic Test Scores 

The growing use of math and reading test scores to 
measure school and student performance, spurred by 
the passage of the No Child Left Behind Act (NCLB), 
has heightened an old debate about which compe- 
tencies public schools should encourage students to 
develop. Does the growing focus on academic test 
scores to measure educational success aid schools in 
helping students develop the most important skills 
and abilities? Or might it discourage teachers from 
working with students to enhance other valuable 
skills, habits, and attitudes? 



The answers to these questions depend in part on a 
school’s capacity to help students develop specific 
skills. A more basic issue — and the focus of our 
study — concerns the value of building various 
competencies. We sought to identify competencies 
with substantial effects on students’ later success 
in higher education and the labor market. We also 
estimated how the value of improving each of these 
competencies depends on students’ existing strengths 
and weaknesses. 

Methods, Data, and Measures 

Research has already examined the benefit for the 
average student of improving test scores and some 
other competencies. But this focus on average effects 
can obscure important differences among students. 
Using a flexible model to analyze nonlinear effects 
and interactions among competencies, we estimated 
the effeet of a marginal improvement in each 
competency on postsecondary earnings and 
educational attainment as a function of students’ 
existing competencies. 

We used data colleeted for NELS, which followed a 
cohort of students who were in eighth grade in 1988. 
Our competency measures and control variables were 
based on the NELS high school tests and surveys of 
students, teachers, and parents conducted in 1988, 
1990, and 1992. The 2000 wave of NELS, conducted 
eight years after students were scheduled to complete 
high school, provided the outcome measures of post- 
secondary educational attainment and earnings. Our 
base sample consisted of 9,977 high school graduates. 

Because students’ true competencies cannot be 
observed directly, we analyzed composite variables 
that serve as indicators, or statistical markers, for 
these competencies. We grouped individual NELS 
variables into categories on the basis of the underly- 










COMPETENCIES AND THEIR MEASURES 



\. Academic achievement as measured by math 
test scores. We analyzed the NELS math, reading, 
history, and science tests administered in eighth, 
tenth, and twelfth grades. Our models relied on 
math scores because they are highly correlated with 
scores on the other tests and more strongly related 
to postsecondary education and earnings. (The 
NELS reading test, in particular, is a weak predictor 
of postsecondary outcomes). 

2. Work habits. NELS asked questions of students 
and teachers about students’ work habits. We 
combined these data into one composite. Questions 
focused on how much time students spent on home- 
work, how hard they worked in class, how often 
they came to class with books and other materials, 
and how often they were tardy or absent. 

3. Sports-related competencies. Sports participation 
is associated with competencies such as team- 
work, competitiveness, and ability to set and work 
toward goals. Our primary measure in this area 
was the number of sports a student participated in 
during high school, transformed into a percentile. 
The model measured the effect of competencies 
correlated with sports participation, some of which 
students may possess before participating. 

4. Leadership skills. NELS recorded leadership roles 
students play in extracurricular activities, includ- 
ing student government, sports, and nonathletic 
organizations and clubs. Our measure is the number 
of leadership roles a student had during high school, 
transformed into a percentile. As with sports, this 
measure is an indicator of competencies students 
possessed before the leadership activity, as well as 
skills developed through the activity. 

5. Prosocial behavior. NELS asked students and 
teachers whether students got in trouble for disrupt- 
ing or not following rules. This measure does not 
include questions on illegal activity or expulsion 
from school, because these events are too rare to 
analyze reliably and may be substantively different 
from other items in this composite. 

6. Attitudes toward determinants of success. Students 
who believe that success results from hard work 
rather than good fortune may be more likely to suc- 
ceed both in school and on the job. NELS included 
a locus of control composite designed to measure 
the extent to which students believe success is the 
result of hard work as opposed to luck. 



ing type of competency and then fomed measures for 
each of six competencies (see box). The four academic 
test scores we analyzed — math, reading, science, 
and history — are highly correlated with one another, 
so it was not possible to isolate the effects of the 
skills captured by each test. Instead, we focused on 
the math test because it is the strongest predictor of 
postsecondary education and earnings. 

To control for other factors that affect outcomes, our 
model included variables corresponding to students’ 
backgrounds and school characteristics. The student 
variables included gender, race, socioeconomic 
status, household structure, disabilities, and an 
indicator of whether a student was ever held back. 
The school variables included the proportion of 
students who received a free or reduced price lunch, 
the proportion who enrolled in a college-prep 
curriculum, the number of extracurricular activities 
offered, and a set of variables that measured a 
school’s discipline policies. 

Math Versus Other Competencies 

A key question is which competencies are most 
valuable — in other words, which ones benefit the 
largest numbers of students if improved. For each 
student in our sample, we identified the competency 
that, when raised 10 percentile points, generated the 
largest jump in earnings and likelihood of completing 
a postsecondary program. This analysis compared 
the postsecondary outcomes of students with a higher 
level of a given competency to outcomes for students 
with a lower level. 

Increasing math test scores had the largest effect 
on earnings for a plurality of the students, but most 
students benefited more from improving one of the 
nonacademic competencies (Table 1). For example, 
with respect to earnings eight years after high school, 
increasing math test scores would have been most 
effective for just 33 percent of students, but 67 
percent would have benefited more from improving 
a nonacademic competency. Many students would 
have secured the largest earnings benefit from 
improvements in locus of control (30 percent) and 
sports-related competencies (20 percent). Similarly, 
for most students, improving one of the nonacademic 
competencies would have had a larger effect than 
better math scores on their chances of enrolling in 
and completing a postsecondary program. A large 
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fraction would have benefited most from improve- 
ments in work habits. However, for more than half of 
the students, better math test scores would have had 
the largest effect on the likelihood of completing a 
bachelor’s degree. 



TABLE 1 


PERCENTAGE OF STUDENTS 


FOR WHOM EACH 


COMPETENCY HAS THE GREATEST EFFECT* 


Complete 


Complete 


Improved Postsecondary 


Bachelor’s 


Competency Program 


Degree Earnings 



Math Test Score 


30 


56 


Nonacademic 


Competencies 


70 


44 


Work Habits 


Composite 


43 


31 


Sports-Related 


Competencies 


9 


5 


Prosocial Behavior 
Composite 


5 


6 


Leadership Roles 


5 


1 


Locus of Control 


9 


1 



33 

67 

2 

20 

2 

14 

30 



*Eight years after high school. 



Different Students Have Different Needs 

The benefits to students from gains in each compe- 
tency depended on the mix of competencies they 
possessed at the outset. The benefit of improving 
math test scores appeared to be largest for students 
who were weak in math. For students whose math 
skills were stronger, the benefits of additional gains 
in math depended on their other competencies. In 
particular, those with better work habits were more 
likely to experience substantial benefits from 
further improvements in their math skills. In general, 
students reaped the most benefit from improving 
in areas where they were weak. As such, the recent 
emphasis on improving the academic performance of 
low achievers may be well placed. On the other hand, 
our findings also support the view that children with 
high levels of academic achievement could benefit 
substantially by developing nonacademic skills. 

Considering individual strengths and weaknesses 
when deciding which competencies to improve might 
be a more effective strategy than simply encouraging 
all students to improve the same competencies. In 



Table 2, we present percentage point differences 
in effects between a hypothetical “individualized” 
policy and a hypothetical “one-size-fits-all” policy 
on four postsecondary outcomes. In the case of the 
individualized policy, a different competency — the 
one of greatest value to each student — is increased by 
20 percentile points for each student. In the case of 
the one-size-fits-all policy, the same competency — 
the one with the largest overall average effect — is 
increased by 20 percentile points for each student. 

For example, increasing the competency of greatest 
economic value to each individual student by 20 
percentile points is associated with an average 
increase in earnings of 9.3 percent (not shown). This 
contrasts with an increase in earnings of 5.6 percent 
if math test scores were improved by 20 percentile 
points for each student, a statistically significant 
difference of 3.7 percentage points (the last row in 
Table 2). 



TABLE 2 



PERCENTAGE-POINT DIFFERENCES IN EFFECTS OF 
INDIVIDUALIZED AND ONE-SIZE-FITS-ALL" POLICIES 



Differences in 

Onteome Policy Effects 

Enrollment in a Postsecondary Program 2.0* 

Completion of a Postsecondary Program 1.9* 

Bachelor’s Degree 1 .4 

Earnings 3.7* 

Source: Authors’ calculations using data from NELS. 
*Significantly different from zero at the .05 level, two-tailed test. 

Note: Positive numbers correspond to cases where the individual- 
ized policy is more effective. 



Moving Forward 

Our findings have two implications for educational 
policy and practice. First, the increasing focus on 
academic skills, and particularly skills captured 
by standardized tests, may be misplaced if it leads 
schools and parents to neglect the development of 
valuable nonacademic competencies. Second, an 
individualized approach to setting and pursuing 
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competency objectives for each student could benefit 
students in terms of their future educational attainment 
and earnings. 

Educators are likely to face at least two challenges in 
developing nonacademic competencies: 

• Measuring these competencies objectively is 
difficult. For many measures used in this study, 
we relied on student self-reports or reports from 
teachers. In practice, schools could not depend on 
this type of subjective and informal assessment, 
particularly if they are held accountable for stu- 
dents’ mastery of these competencies. Furthermore, 
little is known about how teachers can develop 
most nonacademic competencies. 

* Schools may find it hard to help individual 
students identify and develop the competencies 
they need most. Expanding the use of individual- 
ized education plans, currently required for stu- 
dents with special needs, is one possible strategy 
schools could use to gauge individual needs. 

The plans could help schools advise parents and 
students on classes and activities that could be 
most beneficial. This approach would require a 
large investment in guidance staff and professional 
development. Additional teacher training, curricu- 



lum changes, and reductions in class size might 
help teachers create customized assignments and 
projects for students. 

Finding ways to help students and schools respond 
to the accelerating demands of our knowledge-driven 
society is a national priority. Despite the challenges 
and costs involved in developing a more individual- 
ized approach to schooling, the investment may be 
worthwhile. Additional research could help clarify 
what types of investments would be most beneficial. 
It could also address the value of a broader range of 
competencies, including ones not captured in our 
study, and illuminate how the benefits of specific 
competencies interact with students’ existing skills, 
attitudes, and interests. Research can also play a 
role in helping educators develop methods for mea- 
suring competencies, particularly ones that cannot 
be evaluated well through standardized tests. Finally, 
future education experiments could help policymakers 
identify cost-effective ways to increase academic and 
nonacademic competencies. 

For more information, contact John Deke, senior researcher, at 
(609) 275-2230, jdeke@mathematica-mpr.com. The full report is 
available at www.mathematica-mpr.com. 
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